Databases are often used to support transactional data processing applications, such as financials, sales, order fulfillment, manufacturing, human resources, and enterprise resource planning applications. A system that implements a transaction application and its associated database (e.g., relational database) is referred to as an online transaction processing (OLTP) system. In an OLTP system, there is a need for fast reads and writes of the transactional data to update many portions of the database simultaneously.
OLTP systems are optimized for processing very small amounts of detailed data, but are generally not suited for analytical tasks that involve ad-hoc analysis of large data amounts. In order to perform queries without negatively impacting the performance of the OLTP system, online analytical processing (OLAP) systems were developed. OLAP systems are optimized for complex analytical and ad-hoc queries with a rapid execution time. OLAP systems retrieve data using pre-calculated data summaries, and are not efficient when handling small numbers of users doing custom analytical processing over very large volumes of non-indexed data.
Running both OLTP and OLAP on the same database has become an efficient way to reduce product total cost of ownership (TCO) and improve performance. However, due to the different types of operations involved, a database architecture that is suitable for OLAP optimization may not be suitable for OLTP. Conversely, a database architecture that is suitable for OLTP may not be suitable for OLAP.
For example, OLTP is characterized by a large number of row-based write operations (e.g., INSERT AND UPDATE), while OLAP typically optimizes performance by compressing attribute (or columns) with the help of dictionaries. The use of column-based databases for OLAP has become quite popular. However, due to the model gap between OLTP row-based operations and the column-based architecture, such column-based databases may be the bottleneck of some operations that are critical to OLTP. This is especially true for OLTP applications that involve many row-based write operations (e.g., INSERT and UPDATE) that make up about 50% of the total transaction time in a typical enterprise resource planning (ERP) transaction (e.g., adding 100-line invoice with serial-and-batch items).
Thus, a need exists for systems, methods, and apparatuses to address the shortfalls of current technology, and to provide other new and innovative features.